Multi-label learning based semi-global matching forest

Semi-Global Matching (SGM) approximates a 2D Markov Random Field (MRF) via multiple 1D scanline optimizations, which serves as a good trade-off between accuracy and efficiency in dense matching. Nevertheless, the performance is limited due to the simple summation of the aggregated costs from all 1D scanline optimizations for the final disparity estimation. SGM-Forest improves the performance of SGM by training a random forest to predict the best scanline according to each scanline’s disparity proposal. The disparity estimated by the best scanline acts as reference to adaptively adopt close proposals for further post-processing. However, in many cases more than one scanline is capable of providing a good prediction. Training the random forest with only one scanline labeled may limit or even confuse the learning procedure when other scanlines can offer similar contributions. In this paper, we propose a multi-label classification strategy to further improve SGM-Forest. Each training sample is allowed to be described by multiple labels (or zero label) if more than one (or none) scanline gives a proper prediction. We test the proposed method on stereo matching datasets, from Middlebury, ETH3D, EuroSDR image matching benchmark, and the 2019 IEEE GRSS data fusion contest. The result indicates that under the framework of SGM-Forest, the multi-label strategy outperforms the single-label scheme consistently

GA-Net-Pyramid: An Efficient End-to-End Network for Dense Matching

Dense matching plays a crucial role in computer vision and remote sensing, to rapidly provide stereo products using inexpensive hardware. Along with the development of deep learning, the Guided Aggregation Network (GA-Net) achieves state-of-the-art performance via the proposed Semi-Global Guided Aggregation layers and reduces the use of costly 3D convolutional layers. To solve the problem of GA-Net requiring large GPU memory consumption, we design a pyramid architecture to modify the model. Starting from a downsampled stereo input, the disparity is estimated and continuously refined through the pyramid levels. Thus, the disparity search is only applied for a small size of stereo pair and then confined within a short residual range for minor correction, leading to highly reduced memory usage and runtime. Tests on close-range, aerial, and satellite data demonstrate that the proposed algorithm achieves significantly higher efficiency (around eight times faster consuming only 20–40% GPU memory) and comparable results with GA-Net on remote sensing data. Thanks to this coarse-to-fine estimation, we successfully process remote sensing datasets with very large disparity ranges, which could not be processed with GA-Net due to GPU memory limitations

Early Detection of Forest Drought Stress with Very High Resolution Stereo and Hyperspectral Imagery

The project ‘Application of remote sensing for the early detection of drought stress at vulnerable forest sites (ForDroughtDet)’ is funded by the German Federal Agency of Agriculture and Food and aims to detect drought stress in an early phase using remote sensing techniques. In this project, three test sites in the south and middle part of Germany are selected. Three levels of observation and analyses are performed. In the first level, close-range stereo images and spectral information are captured with a research crane in Kranzberg forest. In the second level, three study sites are imaged twice in three years by airborne hyperspectral and stereo cameras. In the third level, the drought stress detection approach will be transferred to regional scale by satellite image. In this paper, we will briefly report our results from the first and second levels. In the first level, 3D models of the forest canopies are generated with the MC-CNN based dense matching approaches, with which the 3D shapes of the stressed and healthy trees are analysed. In addition, for the spectral analyses, different chlorophyll-sensitive indices are calculated and compared for the stressed and healthy trees. In order to further analyse the tree drought stress in the second level, a novel individual tree crown (ITC) segmentation approach is proposed and tested on the airborne stereo dataset

Homogeneous pixel selection for distributed scatterers using multitemporal SAR data stacks

Land deformation and topography estimation using Synthetic Aperture Radar (SAR) interferometry has gained more and more attention over the last years due to its high quality. Up to now, there have been many high resolution SAR satellites launched, such as TerraSAR-X, TanDEM-X, and COSMO-SkyMed. Multitemporal interferometric SAR (InSAR) techniques have become important quantitative geodetic tools to monitor deformation time series. The Persistent Scatterer Interferometry (PSI) technique utilizes coherent radar targets exhibiting high phase stability. The technique is reliable for monitoring deformation with millimeter accuracy. Persistent Scatterers (PSs) e.g. man-made structures, boulders, and outcrops are widely available over a city, however, they have less density in non-urban areas. As a result, Distributed Scatterers (DSs) corresponding to image pixels belonging to areas of moderate coherence, where many neighboring pixels share similar reflectivity values (as they belong to the same object), are used in order to increase the density of measurement points, such as in the SqueeSAR, Small BAseline Subset (SBAS) techniques etc. In order to improve the estimation accuracy of interferometric phase and coherence of DSs, multilooking is needed, where neighboring pixels are averaged together with the target pixel to reduce phase noise. Conventional boxcar kernel multilooking technique is based on the hypothesis of statistical homogeneity of the averaged pixels in a rectangular window surrounding the DS pixel. Nevertheless, as the size of the kernel increases, this hypothesis loses its validity. There can be loss of details as pixels arising from different statistical distributions are averaged together. Therefore, an adaptive multilooking technique is required to preserve the high resolution provided by modern satellites. This thesis aims to find reliable and robust methods to select homogeneous pixels, i.e. the pixels belonging to the same category of scatterers, to perform adaptive multilooking of DSs. The focus is on techniques that make use of the amplitude information of a multitemporal SAR data stack. Accordingly, two new methods, based on Confidence Interval (CI) of amplitude mean and CI of amplitude median have been developed as part of the thesis and principles behind these are clarified. Results of the selected methods are shown and compared using TerraSAR-X data from two different test regions (i.e. Lueneburg and Cologne). Performance, quality and robustness are analyzed. At last, the conclusions are summarized and some suggestions are proposed for future research. The algorithms can aid in accurate covariance matrix estimation and can be widely applied in topographic mapping and deformation monitoring

GA-Net-Pyramid: An Efficient End-to-End Network for Dense Matching

Dense matching plays a crucial role in computer vision and remote sensing, to rapidly provide stereo products using inexpensive hardware. Along with the development of deep learning, the Guided Aggregation Network (GA-Net) achieves state-of-the-art performance via the proposed Semi-Global Guided Aggregation layers and reduces the use of costly 3D convolutional layers. To solve the problem of GA-Net requiring large GPU memory consumption, we design a pyramid architecture to modify the model. Starting from a downsampled stereo input, the disparity is estimated and continuously refined through the pyramid levels. Thus, the disparity search is only applied for a small size of stereo pair and then confined within a short residual range for minor correction, leading to highly reduced memory usage and runtime. Tests on close-range, aerial, and satellite data demonstrate that the proposed algorithm achieves significantly higher efficiency (around eight times faster consuming only 20–40% GPU memory) and comparable results with GA-Net on remote sensing data. Thanks to this coarse-to-fine estimation, we successfully process remote sensing datasets with very large disparity ranges, which could not be processed with GA-Net due to GPU memory limitations

Dense matching comparison between census and a convolutional neural network algorithm for plant reconstruction

3D reconstruction of plants is hard to implement, as the complex leaf distribution highly increases the difficulty level in dense matching. Semi-Global Matching has been successfully applied to recover the depth information of a scene, but may perform variably when different matching cost algorithms are used. In this paper two matching cost computation algorithms, Census transform and an algorithm using a convolutional neural network, are tested for plant reconstruction based on Semi-Global Matching. High resolution close-range photogrammetric images from a handheld camera are used for the experiment. The disparity maps generated based on the two selected matching cost methods are comparable with acceptable quality, which shows the good performance of Census and the potential of neural networks to improve the dense matching